The Relevance of Connectionism to AI: A Representation and Reasoning Perspective

It is generally acknowledged that tremendous computational activity underlies some of the most commonplace cognitive behavior. If we view these computations as systematic rule governed operations over symbolic structures (i.e., inferences) we are confronted with the following challenge: Any generalized notion of inference is intractable, yet our ability to perform cognitive tasks such as language understanding in real-time suggests that we are capable of performing a wide range of inferences with extreme efficiency - almost as a matter of reflex. One response to the above challenge is that the traditional formulation is simply inappropriate and it is erroneous to view computations underlying cognition as inferences. An alternate response - and the one pursued in this paper - is that the traditional account is basically sound: The notion of symbolic representation is fundamental to a computational model of cognition and so is the view that computations in a cognitive system correspond to systematic rule governed operations. However, there is much more to a computational account of cognition than what is captured by these assertions. What is missing is an appreciation of the intimate and symbiotic relationship between the nature of representation, the effectiveness of inference, and the computational architecture in which the computations are situated. We argue that the structured connectionist approach offers the appropriate framework for explicating this symbiotic relationship and meeting the challenge of computational effectiveness

A Connectionist System for Rule Based Reasoning With Multi-Place Predicates and Variables

McCarthy has observed that the representational power of most connectionist systems is restricted to unary predicates applied to a fixed object. More recently, Fodor and Pylyshyn have made a sweeping claim that connectionist systems cannot incorporate systematicity and compositionality. These comments suggest that representing structured knowledge in a connectionist network and using this knowledge in a systematic way is considered difficult if not impossible. The work reported in this paper demonstrates that a connectionist system can not only represent structured knowledge and display systematic behavior, but it can also do so with extreme efficiency. The paper describes a connectionist system that can represent knowledge expressed as rules and facts involving multi-place predicates (i.e., n-ary relations), and draw limited, but sound, inferences based on this knowledge. The system is extremely efficient - in fact, optimal, as it draws conclusions in time proportional to the length of the proof. It is observed that representing and reasoning with structured knowledge requires a solution to the variable binding problem. A solution to this problem using a multi-phase clock is proposed. The solution allows the system to maintain and propagate an arbitrary number of variable bindings during the reasoning process. The work also identifies constraints on the structure of inferential dependencies and the nature of quantification in individual rules that are required for efficient reasoning. These constraints may eventually help in modelling the remarkable human ability of performing certain inferences with extreme efficiency

From Simple Associations to Systemic Reasoning: A Connectionist Representation of Rules, Variables and Dynamic Bindings

Human agents draw a variety of inferences effortlessly, spontaneously, and with remarkable efficiency - as though these inferences are a reflex response of their cognitive apparatus. The work presented in this paper is a step toward a computational account of this remarkable reasoning ability. We describe how a connectionist system made up of simple and slow neuron-like elements can encode millions of facts and rules involving n-ary predicates and variables, and yet perform a variety of inferences within hundreds of milliseconds. We observe that an efficient reasoning system must represent and propagate, dynamically, a large number of variable bindings. The proposed system does so by propagating rhythmic patterns of activity wherein dynamic bindings are represented as the in-phase, i.e., synchronous, firing of appropriate nodes. The mechanisms for representing and propagating dynamic bindings are biologically plausible. Neurophysiological evidence suggests that similar mechanisms may in fact be used by the brain to represent and process sensorimotor information

Character Recognition Using A Modular Spatiotemporal Connectionist Model

We describe a connectionist model for recognizing handprinted characters. Instead of treating the input as a static signal, the image is scanned over time and converted into a time-varying signal. The temporalized image is processed by a spatiotemporal connectionist network suitable for dealing with time-varying signals. The resulting system offers several attractive features, including shift-invariance and inherent retention of local spatial relationships along the temporalized axis, a reduction in the number of free parameters, and the ability to process images of arbitrary length. Connectionist networks were chosen as they offer learnability, rapid recognition, and attractive commercial possibilities. A modular and structured approach was taken in order to simplify network construction, optimization and analysis. Results on the task of handprinted digit recognition are among the best report to date on a set of real-world ZIP code digit images, provided by the United States Postal Service. The system achieved a 99.1% recognition rate on the training set and a 96.0% recognition rate on the test set with no rejections. A 99.0% recognition rate on the test set was achieved when 14.6% of the images were rejected

Massively Parallel Simulation of Structured Connectionist Networks: An Interim Report

We map structured connectionist models of knowledge representation and reasoning onto existing general purpose massively parallel architectures with the objective of developing and implementing practical, real-time knowledge base systems. Shruti, a connectionist knowledge representation and reasoning system which attempts to model reflexive reasoning, will serve as our representative connectionist model. Efficient simulation systems for shruti are developed on the Connection Machine CM-2 - an SIMD architecture - and on the Connection Machine CM-5 - an MIMD architecture. The resulting simulators are evaluated and tested using large, random knowledge bases with up to half a million rules and facts. Though SIMD simulations on the CM-2 are reasonably fast - requiring a few seconds to tens of seconds for answering simple queries - experiments indicate that MIMD simulations are vastly superior to SIMD simulations and offer hundred- to thousand-fold speedups. This work provides new insights into the simulation of structured connectionist networks on massively parallel machines and is a step toward developing large yet efficient knowledge representation and reasoning systems

Incremental Class Learning Approach and Its Applications to

Abstract Incremental Class Learning (ICL) provides a feasible framework for the development of scalable learning systems. Instead of learning a complex problem at once, ICL focuses on learning subproblems incrementally, one at a time -using the results of prior learning for subsequent learning -and then combining the solutions in an appropriate manner. With respect to multi-class classification problems, the ICL approach presented in this paper can be summarized as follows. Initially the system focuses on one category. After it learns this category, it tries to identify a compact subset of features (nodes) in the hidden layers, that are crucial for the recognition of this category. The system then freezes these crucial nodes (features) by fixing their incoming weights. As a result, these features cannot be obliterated in subsequent learning. These frozen features are available during subsequent learning and can serve as parts of weight structures built to recognize other categories. As more categories are learned, the set of features gradually stabilizes and learning a new category requires less effort. Eventually, learning a new category may only involve combining existing features in an appropriate manner. The approach promotes the sharing of learned features among a number of categories and also alleviates the wellknown catastrophic interference problem. We present promising results of applying the ICL approach to the unconstrained Handwritten Digit Recognition problem, based on a spatio-temporal representation of patterns

A Lower Bound Result for the Common Element Problem and Its Implication for Reflexive Reasoning

In this paper we prove a lower bound of Ω(n log n) for the common element problem on two sets of size n each. Two interesting consequences of this lower bound are also discussed. In particular, we show that linear space neural network models that admit unbalanced rules cannot draw all inferences in time independent of the knowledge base size. We also show that the join operation in data base applications needs Ω(log n) time given only n processors

A Connectionist model of Planning via Back-chaining Search

A connectionist model for emergent planning behavior is proposed. The model demonstrates that a simple planning schema, acting in concert with two general purpose cognitive functionalities, namely, episodic memory and perception, can solve a restricted class of planning problems by backchaining from the goal to the current state. In spite of its simple structure, the schema can search for and execute plans involving multiple steps. We discuss how this simple model can be extended into a more powerful and expressive planning system by incorporating additional control and memory structures

Suggestion Mining from Customer Reviews

The increasing online content has influenced users’ buying behavior. It has triggered a paradigm shift in marketing strategies,as the consumer is no longer swayed by marketers, instead relying on user comments for a particular product or service. Thispaper focuses on extracting information from feedbacks like suggestions and recommendation by the users that is oftenpresent along with the sentiment. While Sentiment Analysis looks at extraction of consumer sentiment, our focus is onextracting actionable feedback present in the text for use by different stakeholders like business analysts and the customer.Our focus is on mining the key suggestions present in text which would benefit the product developer. We present our resultsand observations in the paper